Sharing virtual content in a mixed reality scene

ABSTRACT

A method of sharing and a method of presenting virtual content in a mixed reality scene rendered on at least two user devices having different viewing position and/or orientation onto the mixed reality scene and corresponding apparatus are described. At a first user device, a user is enabled to select a virtual content to be shared and a second user device with whom the virtual content is to be shared. Information related to the virtual content to be shared is provided, wherein the provided information comprises the 3D position of the virtual content to be shared. The information is received by the second user device and the shared virtual content is rendered with regard to the viewing position and/or orientation of the second user device onto the mixed reality scene.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a continuation application of U.S. patentapplication Ser. No. 17/253,543, entitled “SHARING VIRTUAL CONTENT IN AMIXED REALITY SCENE,” filed Dec. 17, 2020, which is incorporated hereinby reference in its entirety and which is a national stage applicationunder 35 U.S.C. § 371 of International Application No.PCT/EP2019/065521, entitled “SHARING VIRTUAL CONTENT IN A MIXED REALITYSCENE,” filed on Jun. 13, 2019, which claims benefit from EuropeanPatent Application Serial No. 18305761.1, entitled “SHARING VIRTUALCONTENT IN A MIXED REALITY SCENE,” filed Jun. 19, 2018.

TECHNICAL FIELD

The present principle relates to a method of sharing virtual content ina mixed reality scene rendered on a first user device and a method ofpresenting the shared virtual content on at least a second user device,wherein the at least two devices have different viewing positions and/ororientations onto the mixed reality scene.

BACKGROUND

This section is intended to introduce the reader to various aspects ofart, which may be related to the present embodiments that are describedbelow. This discussion is believed to be helpful in providing the readerwith background information to facilitate a better understanding of thevarious aspects of the present disclosure. Accordingly, it should beunderstood that these statements are to be read in this light.

The significant increase in performance and speed of computers, graphicscards and processors, the evolution of displays, and the development ofcompletely new mobile devices have resulted in many applications forVirtual Reality (VR) and Mixed Reality (MR) technologies, both beingpart of the so-called Mediated Reality. MR is a mix of reality andvirtual reality, i.e. the merging of real and virtual content,encompassing both Augmented Reality (AR) and Augmented Virtuality (AV).

VR enables a user to experience a virtual, surrounding 3D digital worldwith suitable tools, to view it from different sides, to move around init and possibly even to interact with it. In the VR environment, theuser no longer perceives his real environment or only to a very limiteddegree.

AR is an enrichment of the real world perceived by the user throughvirtual content, especially in the form of texts, graphics, animations,videos, static or moving 3D objects. The user typically perceives thevirtual content on a display where it is merged into the real world.Correspondingly, AV refers to the merging of real world content into apredominantly virtual world.

For the presentation of MR/VR content, different devices can be usedsuch as mobile devices having a display like mobile phones or computertablets. In this case, a MR presentation is made possible by displayingthe virtual content on the display together with the real environmentrecorded by a camera of the respective device. Another example for adevice displaying MR/VR content is a head-mounted display (HMD), whichis worn by a user like normal glasses, but has one or more projection ordisplaying units to project images in front of the eyes or directly onthe retina. Depending on whether the head-mounted display is for VR orMR content, the wearer can perceive only the virtual surrounding 3Ddigital world or also the surrounding real environment. As anotherexample, AR content can also be displayed by a projection of virtualcontent on transparent surfaces that allow a view of the real world,such as windows.

While for many of today's applications the MR/VR content is onlypresented to a single user, there is also an increasing interest insimultaneous participation of several users in the same MR/VR scene andto share between these users a virtual content contained in the MR/VRscene.

A method for sharing mediated reality content is disclosed in WO2017/021587 A1. The mediated reality content is displayed by a deviceand a further device is identified with which at least a portion of themediated reality content is to be shared. One or more parameterscharacterizing a display associated with the further device and aportion of the mediated reality content to be shared with the furtherdevice are determined. The portion of the mediated reality content isshared with the further device.

SUMMARY

In a first aspect, a method of sharing virtual content in a mixedreality scene rendered on at least two user devices having differentviewing positions and/or orientations onto the mixed reality scene isdescribed. The method comprises, at a first user device:

-   -   enabling a user to select a virtual content to be shared and a        second user device with whom the virtual content is to be        shared; and    -   providing information related to the virtual content to be        shared with the second user device, wherein the provided        information comprises the 3D position of the virtual content to        be shared.

In a second aspect, a method of presenting virtual content in a mixedreality scene rendered on at least two user devices having differentviewing positions and/or orientations onto the mixed reality scene isdescribed. The method comprises, at a second user device:

-   -   receiving from a first user device information related to a        virtual content in the mixed reality scene, which is shared by        the first user device with the second user device, wherein the        received information comprises the 3D position of the virtual        content to be shared; and    -   rendering the shared virtual content with regard to the viewing        position and/or orientation of the second user device onto the        mixed reality scene.

In a third aspect, an apparatus for sharing virtual content in a mixedreality scene rendered on at least two user devices having differentviewing positions and/or orientations onto the mixed reality scene isdescribed. The apparatus comprises:

-   -   means for enabling a selection of a virtual content to be shared        and of a second user device with whom the virtual content is to        be shared; and    -   means for providing information related to the virtual content        to be shared to the second user device, wherein the provided        information comprises the 3D position of the virtual content to        be shared.

In a fourth aspect, an apparatus for presenting virtual content in amixed reality scene rendered on at least two user devices havingdifferent viewing positions and/or orientations onto the mixed realityscene is described. The apparatus comprises:

-   -   means for receiving from a first user device information related        to a virtual content in the mixed reality scene which is shared        by the first user device with the second user device, wherein        the received information comprises the 3D position of the        virtual content to be shared; and    -   means for rendering the shared virtual content with regard to        the viewing position and/or orientation of the second user        device onto the mixed reality scene.

The above presents a simplified summary of the subject matter in orderto provide a basic understanding of some aspects of subject matterembodiments. This summary is not an extensive overview of the subjectmatter. It is not intended to identify key/critical elements of theembodiments or to delineate the scope of the subject matter. Its solepurpose is to present some concepts of the subject matter in asimplified form as a prelude to the more detailed description that ispresented later.

BRIEF DESCRIPTION OF THE DRAWINGS

The proposed method and apparatus is best understood from the followingdetailed description when read in conjunction with the accompanyingdrawings. The drawings include the following figures briefly describedbelow:

FIG. 1 illustrates a flowchart of an exemplary method of sharing virtualcontent in a mixed reality scene in accordance with one embodiment ofthe present disclosure.

FIG. 2 illustrates a flowchart of an exemplary method of presenting theshared virtual content on a second user device in accordance with oneembodiment of the present disclosure.

FIG. 3A illustrates three MR rendering devices with an exchange ofsharing information between two of them.

FIG. 3B illustrates three MR rendering devices with an exchange ofsharing information between two of them.

FIG. 3C illustrates three MR rendering devices with an exchange ofsharing information between two of them.

FIG. 4A illustrates three MR rendering devices and a server with anexchange of sharing information between three MR devices via the server.

FIG. 4B illustrates three MR rendering devices and a server with anexchange of sharing information between two MR devices via the server.

FIG. 4C illustrates three MR rendering devices and a server with anexchange of sharing information between two MR devices via the server.

FIG. 5 illustrates a first and a second user, each observing the sameenvironment with their own user device.

FIG. 6A illustrates a method for selecting both a content to be sharedin a displayed MR scene and a user with whom to share using atouchscreen display.

FIG. 6B illustrates a method for selecting both a content to be sharedin a displayed MR scene and a user with whom to share using atouchscreen display.

FIG. 7A illustrates an example of on screen information displayed to thesecond user notifying him that another user wants to share content withhim.

FIG. 7B illustrates an example of on screen information displayed to thesecond user notifying him that another user wants to share content withhim.

FIG. 7C illustrates an example of on screen information displayed to thesecond user notifying him that another user wants to share content withhim.

FIG. 8 illustrates a rendered MR scene shown on the display of a seconduser device after the second user has accepted the sharing.

FIG. 9 illustrates an animation to generate an appearing effect for theshared content.

FIG. 10A illustrates an example of the rendering of a MR scene displayedon the first user device with an acceptance notification or usinghighlighting for notifying the first user.

FIG. 10B illustrates an example of the rendering of a MR scene displayedon the first user device with an acceptance notification or usinghighlighting for notifying the first user.

FIG. 11 illustrates a view on the MR scene, wherein some users arerepresented by icons.

FIG. 12 illustrates a block diagram of an apparatus that is configuredto enable sharing of mixed or virtual reality content.

It should be understood that the drawings are for purposes ofillustrating the concepts of the disclosure and are not necessarily theonly possible configuration for illustrating the disclosure.

DETAILED DESCRIPTION

All examples and conditional language recited herein are intended foreducational purposes to aid the reader in understanding the principlesof the disclosure and the concepts contributed by the inventor tofurthering the art, and are to be construed as being without limitationto such specifically recited examples and conditions.

It should be understood that the elements shown in the figures may beimplemented in various forms of hardware, software or combinationsthereof. Preferably, these elements are implemented in a combination ofhardware and software on one or more appropriately programmedgeneral-purpose devices, which may include a processor, memory andinput/output interfaces. Herein, the phrase “coupled” is defined to meandirectly connected to or indirectly connected with through one or moreintermediate components. Such intermediate components may include bothhardware and software based components.

It will be appreciated by those skilled in the art that the flowdiagrams represent various processes, which may be executed by acomputer or processor, whether or not, such computer or processor isexplicitly shown. Similarly, it will be appreciated that the blockdiagram presented herein represents a conceptual view of an illustrativecircuitry embodying the principles of the disclosure.

The functions of the various elements shown in the figures may beprovided through the use of dedicated hardware as well as hardwarecapable of executing software in association with appropriate software.When provided by a processor, the functions may be provided by a singlededicated processor, by a single shared processor, or by a plurality ofindividual processors, some of which may be shared. Other hardware,conventional and/or custom, may also be included. Similarly, anyswitches shown in the figures are conceptual only. Their function may becarried out through the operation of program logic, through dedicatedlogic, through the interaction of program control and dedicated logic,or even manually, the particular technique being selectable by theimplementer as more specifically understood from the context.

In the claims hereof, any element expressed as a means for performing aspecified function is intended to encompass any way of performing thatfunction including, for example, a) a combination of circuit elementsthat performs that function or b) software in any form, including,therefore, firmware, microcode or the like, combined with appropriatecircuitry for executing that software to perform the function.

In an embodiment, a method of sharing virtual content in a mixed realityscene rendered on at least two user devices having different viewingpositions and/or orientations onto the mixed reality scene may include,at a first user device:

-   -   enabling a user to select a virtual content to be shared and a        second user device with whom the virtual content is to be        shared; and    -   providing information related to the virtual content to be        shared to the second user device, wherein the provided        information comprises the 3D position of the virtual content to        be shared.

In some embodiments, the method may also include:

-   -   rendering the content to be shared on the first user device        using the viewing position and/or orientation of the first user        device and the 3D position of the content to be shared.

In some embodiments, said enabling may also include receiving a userinput for selecting the content to be shared in the rendered mixedreality scene.

In some embodiments, the mixed reality scene rendered on the first userdevice may also include the second user device and/or a user of thesecond user device and/or a graphical symbol representing the seconduser device or user and said enabling may also include detecting a userinput for selecting the rendered user device, user or graphical symbolin the rendered mixed reality scene.

Some embodiments may also include that the mixed reality scene isrendered on a touchscreen of the first user device and that the detecteduser input is a gesture performed by a user on the touchscreen.

In some embodiments, the performed gesture may also include:

-   -   selecting the content to be shared by touching the touchscreen        at the location of the displayed content to be shared;    -   dragging the selected content to the location of the rendered        user device, user or graphical symbol; and    -   sharing the selected content by dropping it onto the rendered        user device, user or graphical symbol.

In some embodiments, the method may also include that the second userdevice and/or user of the second user device is represented by a 3Dshape.

Turning now to FIG. 1 , a flowchart of an exemplary method of sharingvirtual content in a mixed or virtual reality scene is shown anddiscussed in detail. In general, the virtual content to be shared canencompass any kind of signal, which can affect the human body and can beperceived by a person through the sense organs, like visual, audible ortactile signals.

However, since visual perceptible content is most common in today's MRNR content, in the following description the virtual content to beshared represents such displayable content.

The method is performed by a first user device of a first user. Withreference to FIG. 5 , the first user device is in the following alsolabeled with DA, the first user with UA, and similarly a second userdevice with DB and a second user with UB.

The first user device can be any kind of MR device, which allows therendering of visual content, for example, a mobile phone, a tabletcomputer, or a head mounted device.

Initially, in step 1 the first user device DA renders a MR scene, whichis displayed on a display of the first user device DA so that it can beviewed by the first user UA. The MR scene can be any kind of mixturebetween real and virtual content, an example of which is given below inFIG. 8 . Instead of performing the rendering in the first user deviceDA, part or all of the rendering could also be performed remotely by arendering server or rendering farm.

The first user UA now personalizes the MR scene. Such a personalizationcan encompass for example one or more of the following amendments ofparts of the scene: change of the 3D location, the 3D orientation, thescale, the geometry, the color, or the texture of a virtual object.Furthermore, new virtual content can be created or downloaded orexisting content can be removed. The processed virtual content could beany kind of visual perceptual content like a 3D graphic object, a 2Dimage, text, or a video clip. The personalized MR scene is rerendered instep 2 so that the first user UA can look at the amended MR scene.

Once satisfied with the personalized MR scene, the first user UA maywant to allow one or more other users having their own MR device andhaving access to same MR scene to view the personalized MR scene, i.e.the first user UA may want to share the processed virtual content. Thefirst user UA and the one or more other users could all physically be inthe same room, viewing and possibly collaborating on the same MR scenebut could also be in different rooms, in particular in a VR environment.All users may simultaneously personalize their own version of the sceneand decide if they want to share, when they want to share, if they wantto share all or only part of the personalized scene, and whether toshare with one of, some of or all the other users. However, for the sakeof simplicity, in the following only a single other user, a second userUB using a second user device DB, is considered.

In particular, if the first user UA has made several amendments to theMR scene, and only wants to share a part of the amendments with seconduser UB, he has to select the piece of content he wants to share. Forthis purpose, the first user UA has to input the selection of thecontent to be shared, which is received by his user device DA in step 3.While various input means for receiving the user input can beimplemented in the user device, the use of a touchscreen is particularlysuitable for mobile devices such as a mobile phone or a tablet computer.In case of a head-mounted display, the user input can be detected byusing head and eye tracking technology.

If there is more than one other user and the first user UA does not wantto share the personalized content with all the other users, he has toidentify and select the user device(s) he wants to share the contentwith. The corresponding input of first user UA is received by his userdevice DA in step 4, wherein the same input means as for the precedingstep 3 can be used.

While in the above the first content to be shared has been selected instep 3, followed by the selection of the second user device with whom toshare in step 4, the selection could also be performed in the reverseorder.

After the first user UA has selected both the content to be shared andthe other user device DB, his user device DA sends in step 5 anotification to the other user device DB, either directly or via aserver, and then waits for a response from user device DB.

In the following step 6, user device DA checks whether an acceptancenotification is received from the other user device DB, wherein theacceptance notification is a confirmation that user UB has accepted thesharing and may include a request for various kinds of informationrelated to the specific content to be shared. If an acceptancenotification is received, user device DA sends in step 7 directly or viathe server the required information, which can be one or more of thefollowing data: an identifier that identifies the object to be shared,the location, the orientation, the scale, the color, or the texture ofthe virtual content.

Furthermore, in step 8, a notification can be given to the first user UAinforming him that the other user UB now sees the shared content or thatuser UB declined the sharing, e.g. by displaying a corresponding messageon the display of user device DA. After that, the method can return tostep 2 in order to allow further sharing of additional contentpersonalized by the first user UA. The method also continues in step 2if instead of an acceptance notification a refusal notification isreceived from user device DB, or if no notification is received at allwithin a certain time period after sending the notification to the otheruser device DB in step 5.

In an embodiment, a method of presenting virtual content in a mixedreality scene rendered on at least two user devices having differentviewing positions and/or orientations onto the mixed reality scene mayinclude, at a second user device:

-   -   receiving from a first user device information related to a        virtual content in the mixed reality scene which is shared by        the first user device with the second user device, wherein the        received information comprises the 3D position of the virtual        content to be shared; and    -   rendering the shared virtual content with regard to the viewing        position and/or orientation of the second user device onto the        mixed reality scene.

In some embodiments, the method may also include:

-   -   receiving a notification that the first user device wants to        share virtual content;    -   detecting a user input accepting or declining the sharing of the        virtual content; and wherein the shared virtual content is only        rendered if a user input accepting the sharing is detected.

In some embodiments the method may also include that the at least twouser devices exchange information via a server.

In some embodiments, the method may also include that a user device,when being turned on, requests from the server at least one of thefollowing data:

-   -   a scene configuration of the mixed reality scene;    -   position data for the current location of the other user        devices; or    -   a list of sharable virtual content.

FIG. 2 illustrates a flowchart of an exemplary method of presenting theshared virtual content on a second user device in accordance with oneembodiment of the present disclosure. The method is performed by asecond user device DB of the second user UB. Like the first user deviceDA, also the second user device DB can be any kind of MR device, whichallows the rendering of visual content, and can be of the same type orof different type as the first user device.

Also, this method starts with a rendering step 9 of a MR scene, which isdisplayed on a display of the second user device DB so that it can beviewed by the second user UB. The MR scene can be the same as the onealso originally rendered on the first user device DA of the first userUA, i.e. before any personalization. Just as well, the second user UBcan also have personalized the original MR scene already on his part. Onthe other hand, content already personalized by the first user UA hasnot yet been communicated to the second user device DB and thus is nottaken into account in rendering step 9.

The second user UB now receives in step 10 a sharing notification fromthe first user device DA, informing him that the first user UA wants toshare content being part of a personalization of the MR scene by thefirst user UA. This notification can be displayed on the screen of thesecond user device DB as shown in FIG. 7 and explained in more detail inthe corresponding description. Instead of or in addition to displayingsuch a notification message, also other means for informing the seconduser UB can be used such as an illuminated LED, a voice output orcausing the second user device to vibrate.

In the following step 11, the second user device DB now obtains an inputby the second user UB, accepting or declining the offer to receive thepersonalized content of the first user UA. Various input means can beused for that, like a touch screen on which icons for accepting ordeclining the offer are displayed, manual input elements like buttons,or a voice control unit.

The second user device DB then checks in step 12 whether the commandreceived from the second user is a command to accept or refuse thesharing offer.

If an input to accept the sharing offer has been obtained, user deviceDB sends in step 13, directly or via a server, a request for informationrequired for rendering the content to be shared.

In the following step 14, the requested information is received by userdevice DB. Once the user device DB has received all shared information,the corresponding virtual content is created if it was missing so farfrom the MR scene observed by user UB, or updated according to thepersonalization by the first user UA, if it was already present in theMR scene observed by user UB. Subsequently, the updated MR scene isrendered on user device DB with the content shared by user device DA,taking into account the point of view of the second user UB on the MRscene and the environment.

Finally, in step 15, a notification is sent to the first user device DAwith an information that the shared content is rendered on the seconduser device DB or that the second user UB has declined the sharing.After that, the method can return to step 10 so that further sharingnotifications can be received.

FIGS. 3A-3C illustrate three MR rendering devices DA, DB, DC with anexchange of sharing information between two of them. The number of MRrendering devices is just an example, there could be various numbers ofMR rendering devices and an exchange of information between more thantwo of them. Furthermore, even though the MR rendering devices are alldepicted in the figure by the same graphical representation, they can beimplemented in different technical devices, for example, the first onein a mobile phone, the second one in a tablet computer, and the thirdone in a head mounted device. These devices exchange information bywireless communication such as Wi-Fi or through a wired connection.

As depicted in FIG. 3A, each of the three MR rendering devices exchangesinformation such as the current 3D location, the configuration of the MRscene or a list of sharable contents with the other two MR renderingdevices. The 3D location is given by a (X, Y, Z) triplet in the “worldcoordinates system” of the MR scene, which can be determined bypositional tracking technologies. The information can be exchanged, whenthe respective MR rendering device is turned on, either completely oronly partially, with a completion when specifically requested or anupdate of information in regular or irregular time intervals.

If a first user UA now wants to share content with a second user UB, hisfirst user device DA sends a sharing notification to the second userdevice DB, as depicted in FIG. 3B. Since the first user has no intentionto share the content with a user of a third user device DC, no sharingnotification is sent to the third user device DC. In case that a previewof the content to be shared shall be rendered on the second user deviceDB, the information sent to the second user device DB has to includeinformation related to the virtual content to be shared like anidentifier of a 3D model, the location, orientation, scale, a texture IDor color information. Furthermore, if the 3D model ID and/or the textureID are not recognized by the second user device DB, a low-resolutionmodel and a low-resolution texture map can be sent too.

FIG. 3C shows the data exchange between the first user device DA and thesecond user device DB once sharing has been accepted. Following arequest from the second user device DB for information related to thecontent to be shared, the first user device DA retrieves the requestedinformation and sends it back to the second user device DB. If therequested information includes a 3D model ID and a texture ID, which isnot recognized by the second user device DB, a full-resolution model anda full-resolution texture map should be sent too.

Instead of directly communicating with each other, the MR renderingdevices can also exchange sharing information via a server asillustrated in FIGS. 4A-4C. Again, the MR rendering devices can bedifferent technical devices and can be connected wireless or wired. Theserver may be for example a computer or a cluster of computers, aset-top-box, or a gateway and can also communicate wireless or wiredwith the MR rendering devices.

As depicted in FIG. 4A, three MR rendering devices DA, DB, DC exchangeinformation through the server S. Each of the three MR rendering devicessends requests for information such as the current location, theconfiguration of the MR scene or a list of sharable contents to theserver, e.g. when a particular MR rendering device is turned on. Inreturn, the server sends the requested information.

As illustrated in FIG. 4B, for sharing content from a first user deviceDA with a second user device DB, a sharing notification and informationon the content to be shared is sent from the first user device DA to theserver S. The server S then forwards the sharing notification to thesecond user device DB. The information on the content to be shared istemporarily stored at the server and not yet forwarded since it is notyet known whether the second user will accept the sharing offer. In thisway, unnecessarily occupying transmission bandwidth and wasting memoryspace of the second user device DB for content, which finally will notbe rendered at the second user device DB, can be avoided.

FIG. 4C shows the data exchange between the first user device DA and thesecond user device DB via the server S if the sharing has been acceptedby the second user. In this case, the second user device DB sends arequest for the required information related to the content to be sharedto the server, which now forwards the buffered information on thecontent to be shared to the second user device DB. Furthermore, theserver S sends a notification about the acceptance or refusal of thesharing to the first user device DA.

An example for a situation where two users observe the same MR scenefrom different viewing positions with their respective user devices isillustrated in FIG. 5 . The users UA and UB are physically in the sameroom, enjoying a MR experience that allows them to virtually adddifferent models of armchairs to a real set consisting of two armchairsand a coffee table using their MR rendering devices DA and DB.

FIGS. 6A-6B illustrate an embodiment, which conveniently allows toselect both a content to be shared and a user with whom to share using atouchscreen display. In this example, the rendered view of the MR sceneon the display D of the touchscreen of the user device of the first useris shown. The first user has personalized his scene by virtually addingan armchair PA. If the user wants to share this virtually added armchairPA, he can select it simply by pointing with his index finger on it, asshown in FIG. 6A.

The second user UB carrying his user device DB is also depicted on thedisplay D. The point in the middle of the user device DB symbolizes theprojection of the 3D location (X, Y, Z) of the camera of the user deviceDB. For sharing the virtually added armchair PA with the second user UB,the first user now moves his fingertip on the touchscreen to thedepiction of the second user device DB, as shown in FIG. 6B, and dropsit by removing the finger at this position.

In that way, using the camera pose of the first user device DA and itsintrinsic parameters, the 3D location of the content to be shared andthe camera pose of the second user device DB, and using well-known 3Dprojections, the selection and dragging of a sharable content and itsdropping onto the second user device DB can be detected. One way toestimate, which element is being touched, is using so-called raycastingtechniques.

Since it might be difficult to do the dropping exactly at the point ofthe 3D location of the camera of the second user device DB, an area fordropping A around it can be defined. The area for dropping may be a 3Dshape, e.g. a sphere or a cube, with a dimension suitable for making aninteraction easier. To point this out to the first user an indication ofthe area for dropping can be shown to the first user on the display ofthe first user device.

When the final position of the finger is detected as being inside theprojection of the sphere/cube containing the camera of the second userdevice DB, the first user device DA sends a notification to the deviceDB or to the server as mentioned above to inform the second user UB thatthe first user UA wants to share content with him.

FIGS. 7A-7C illustrate some examples of on screen information displayedto the second user UB notifying him that another user wants to sharecontent with him. In the shown examples, the second user UB has himselfalso virtually added a piece of content PB, which is displayed in hisview of the MR scene and is also in the form of an armchair.

The second user device DB generates a virtual user interface UI, whichis composed of various layers of information. To notify the second userUB that the user UA wants to share a piece of content PA with him, anadditional layer is created and displayed onto the screen of the deviceDB with “ACCEPT” and “DECLINE” buttons BA, BD to accept the sharing ornot, as shown in FIG. 7A. This can be as simple as a 2D window displayedin the plane of the screen.

In another embodiment depicted in FIG. 7B a preview PR of the content tobe shared is displayed simultaneously with the “ACCEPT”/“DECLINE”buttons. This preview of the content can be of low-resolution and can bedisplayed in a semi-transparent way.

In an again other embodiment as shown in FIG. 7C, in a first step only athird button BP labeled “PREVIEW” is inserted between the “ACCEPT” and“DECLINE” buttons. In that case, the low-resolution and semi-transparentpreview of the content to be shared is only displayed, if the “PREVIEW”button is pressed. After pressing the “PREVIEW” button, the preview isadded, the “PREVIEW” button disappears and the “ACCEPT” and “DECLINE”buttons remain visible in order to allow the user to still decidewhether to accept the sharing or not based on the displayed MR sceneincluding the preview.

In an again other embodiment, which is not shown in FIGS. 7A-7C, the 3Dlocation of the content to be shared as well as the 3D location of thefirst user UA are shared with the second user device DB, and thenprojected onto the view of the second user device DB. Both projectedlocations are highlighted for some time, indicating to the second userUB where the first user UA and the content to be shared are. If thelocations to be projected are out of the image boundaries, orientedarrows can be displayed close to the image boundaries.

FIG. 8 illustrates the rendered MR scene shown on the display of thesecond user device DB after the second user UB has accepted the sharing.The view now includes besides the real two armchairs and the coffeetable, the virtually added armchair PB, inserted by himself, and theshared virtual armchair PA, inserted by the first user UA into the MRscene.

FIG. 9 illustrates an embodiment, in which the virtual content is notdirectly rendered with its final features, but an animation is createdto generate an appearing effect. For example, for few seconds, a 3Dtrajectory from the location of the first user UA to the final locationof the shared object PA is interpolated with successive displays alongthe trajectory. In the shown example, the 3D trajectory is a translationand the first user UA is not in the field of view of the second user UB.The translation starts from the right boundary of the display indicatingto the second user UB that the first user UA is on his right. However,even though the 3D trajectory is symbolized in the figure by an arrow,it is not restricted to a translation but could have any curved shape.During the animation, the scale of the shared object can remainunchanged or can be interpolated from zero to the final scale.

FIGS. 10A-10B show embodiments of modifications of the graphical userinterface displayed to the first user UA on the first user device DA. Inthe example depicted in FIG. 10A, a window with an on screennotification is displayed for some time on the screen of the first userdevice DA, having text notifying the first user UA that the second userUB accepted the sharing and now can view the MR scene including theshared content. Instead of overlaying a window, the acceptance orrefusal can also be made visible to the first user UA by highlightingthe second user device DB and/or the content to be shared for some time,as shown in FIG. 10B. For example, one or both of them can be given acertain color, like green if the sharing is accepted and red ifdeclined. Using the known 3D location of the second user device DB andthe content to be shared, one or both of them can be approximated by a3D shape.

If the same content shall be shared with several users, this can be donesimultaneously in a single gesture. In one embodiment; this can be doneconveniently just by dragging the content to be shared over to all theusers the content shall be shared with. In another embodiment, for thispurpose the first user can form a closed contour so that every userwhose projected position is inside this closed contour will receive thenotification. To share the content in a single gesture with all userssimultaneously, even including those who are not in sight, the firstuser can form a closed contour by dragging the content to the fourcorners of his screen.

In a further embodiment illustrated in FIG. 11 , some or all users canbe represented by small icons IC in a corner of the screen. In thedepicted example, the second user UB is in sight, i.e. shown on thedisplay of the first user device, whereas three further users C, D and Esharing the same MR scene are not visible and are, therefore,represented by the icons IC. By a simple drag & drop action on one ofthe icons, sharing of the content with the associated user can belaunched. This variant allows to deal easily with the cases where someusers are too far away from the first user and therefore the projectionof their respective user device is too small, or where some users areoccluded or not in the field of view. The display of such icons couldeven be dynamic, i.e. at a time instant the only displayed icons are theones corresponding to users too far away or not visible.

In a further variant, the users are not all physically in the same room,but telepresence is used so that virtual clones of the remote users aredisplayed in the room. Furthermore, if in a VR environment some userswear a VR headset, the users may be represented by an avatar or anotherelement in the VR scene.

In an embodiment, an apparatus for sharing virtual content in a mixedreality scene rendered on at least two user devices having differentviewing positions and/or orientations onto the mixed reality scene, mayinclude:

-   -   means for enabling a user to select a virtual content to be        shared and a second user device with whom the virtual content is        to be shared; and    -   means for providing information related to the virtual content        to be shared to the second user device, wherein the provided        information comprises the 3D position of the virtual content to        be shared.

In some embodiments, the apparatus may also include:

-   -   means for rendering the content to be shared on the first user        device using the viewing position and/or orientation of the        first user device and the 3D position of the content to be        shared.

In some embodiments, the apparatus may also include:

-   -   a touchscreen for displaying the rendered mixed reality scene        including the content to be shared and for detecting a user        input being a gesture performed by a user on the touchscreen.

In an embodiment, an apparatus for presenting virtual content in a mixedreality scene rendered on at least two user devices having differentviewing positions and/or orientations onto the mixed reality scene, mayinclude:

-   -   means for receiving from a first user device information related        to a virtual content in the mixed reality scene which is shared        by the first user device with the second user device, wherein        the received information comprises the 3D position of the        virtual content to be shared; and    -   means for rendering the shared virtual content with regard to        the viewing position and/or orientation of the second user        device onto the mixed reality scene.

A block diagram of an embodiment of an apparatus that is configured toenable sharing of mixed or virtual reality content is illustrated inFIG. 12 . In some embodiments, the apparatus 100 might be a mobiledevice.

The apparatus 100 comprises a processor 101, which for example can be acentral processing unit (CPU) or a graphics processing unit (GPU). Theprocessor 101 is coupled to a memory 102, in which data and/or computerprogram instructions are stored. However, multiple processors and/ormultiple memories may also be provided.

Furthermore, the apparatus 100 comprises a touchscreen display 103coupled to the processor. The touchscreen display 103 allows the user toperceive visually the mixed or virtual reality content and to interactwith the displayed content. The display part of the touchscreen display103 may for example include a liquid crystal display or an organic lightemitting display. For sensing the touch of a user, the touchscreendisplay 103 can use different technologies such as e.g. a resistive orcapacitive touchscreen panel. In other embodiments, instead of thetouchscreen, a display in combination with other input means such asmechanical buttons, a mouse, or a touchpad could be used as well.

A transceiver 104 is also coupled to the processor for a wireless orwired communication with other user devices. The transceiver may forexample include a transmitter and receiver for signal transmissionaccording to a mobile radio standard such as UMTS or LTE, or short rangewireless transmission such as Wi-Fi, Bluetooth.

1-15. (canceled)
 16. A method comprising: enabling, at a first mixedreality (MR) device, a user of the first MR device to select a virtualcontent object displayed by a display of the first MR device; enabling,at the first MR device, the user of the first MR device to select animage of a second MR device visible in the display of the first MRdevice, wherein the selected virtual content object was not previouslyaccessible to the second MR device; and responsive to receiving a userselection of the virtual content object and a user selection of theimage of the second MR device, making the virtual content objectaccessible to the second MR device.
 17. The method according to claim 1,wherein the enabling comprises receiving a user input to select thecontent to be shared in a rendered mixed reality scene.
 18. The methodaccording to claim 17, wherein the mixed reality scene rendered on thefirst MR device comprises the image of the second MR device, and whereinthe image of the second MR device comprises at least one of an image ofthe second MR device itself, an image representative of a user of thesecond MR device, a graphical symbol representing the second MR device,and a graphical symbol representing the user of the second MR device.19. The method according to claim 18, wherein the enabling comprisesdetecting the user input to select the rendered image of the second MRdevice.
 20. The method according to claim 19, wherein the mixed realityscene is rendered on a touchscreen of the first MR device, and whereinthe detected user input is a gesture performed by a user on thetouchscreen.
 21. The method according to claim 20, wherein the performedgesture comprises: selecting the content to be shared by touching thetouchscreen at a location of the displayed content object to be shared;dragging the selected content to the location of the rendered image ofthe second MR device; and sharing the selected content by dropping theselected content onto the rendered image of the second MR device. 22.The method according to claim 18, wherein the image of the second MRdevice is represented by a 3D shape.
 23. The method according to claim16, further comprising: receiving from the first MR device informationrelated to a virtual content in a mixed reality scene which is selectedby the first MR device with the second MR device; and rendering theselected virtual content onto the mixed reality scene with regard to atleast one of a viewing position of the second MR device and anorientation of the second MR device.
 24. The method according to claim23, further comprising: receiving a notification that the first MRdevice wants to share the virtual content; and detecting a user inputaccepting or declining sharing of the virtual content.
 25. The methodaccording to claim 16, wherein the first and second MR devices exchangeinformation via a server.
 26. The method according to claim 25, whereinthe first MR device, when being turned on, requests from the server atleast one of: a scene configuration of a mixed reality scene, positiondata for a current location of another MR device, and a list of sharablevirtual content.
 27. The method according to claim 16, furthercomprising: rendering the selected virtual content object on the firstMR device using at least one of: a viewing position of the first MRdevice, an orientation of the first MR device, and a 3D position of theselected virtual content object.
 28. The method according to claim 16,wherein enabling the user of the first MR device to select the virtualcontent object comprises enabling the user of the first MR device toselect the virtual content object within a mixed reality scene using atouchscreen of the first MR device.
 29. The method according to claim16, wherein enabling the user of the first MR device to select thevirtual content object comprises enabling the user of the first MRdevice to select the virtual content object within a mixed reality sceneusing gaze detection by the first MR device.
 30. The method according toclaim 16, wherein enabling the user of the first MR device to select thevirtual content object and enabling the user of the first MR device toselect the image of the second MR device concurrently comprise enablingthe user of the first MR device to use a drag and drop gesture.
 31. Themethod according to claim 16, wherein enabling the user of the first MRdevice to select the second MR device comprises enabling the user of thefirst MR device to select the image of the second MR device within amixed reality scene using a touchscreen of the first MR device.
 32. Themethod according to claim 16, wherein enabling the user of the first MRdevice to select the image of the second MR device comprises enablingthe user of the first MR device to select the image of the second MRdevice within a mixed reality scene using gaze detection by the first MRdevice.
 33. The method according to claim 16, wherein enabling the userof the first MR device to select the image of the second MR devicecomprises enabling the user of the first MR device to select an icondisplayed in proximity to the image of the second MR device within amixed reality scene displayed on the first MR device.
 34. The method ofclaim 16, further comprising: personalizing, at the first MR device, theselected visual content object, wherein personalizing the selectedvisual content object comprises changing at least one of: a location ofthe selected visual content object, an orientation of the selectedvisual content object, a scale of the selected visual content object, ageometry of the selected visual content object, a color of the selectedvisual content object, and a texture of the selected visual contentobject.
 35. The method of claim 16, wherein the image of the second MRdevice is selected by the user of the first MR device before the visualcontent object is selected by the user of the first MR device.
 36. Themethod of claim 16, further comprising receiving, at the first MRdevice, a notification indicating that the selected visual contentobject is displayed on the second MR device.
 37. The method of claim 16,further comprising receiving, at the first MR device, a refusalnotification indicating that the selected visual content object wasrefused by the user of the second MR device.
 38. The method of claim 16,further comprising determining that no notification was received by thefirst MR device within a timeout period from the second MR deviceregarding the selected visual content object.
 39. The method accordingto claim 16, further comprising: receiving a notification indicatingthat the user of the first MR device wants to share the virtual content,wherein the notification comprises at least one of indication via anilluminated LED, a voice signal, and a vibration of the second MRdevice.
 40. The method of claim 16, further comprising: adding theselected virtual content object to a mixed reality scene, andpersonalizing, at the first MR device, the selected visual contentobject.
 41. The method of claim 40, further comprising sharing, by thefirst MR device, the mixed reality scene with the second MR device. 42.A method comprising: enabling, at a first mixed reality (MR) device,selection of a virtual content object displayed by a display of thefirst MR device; enabling, at the first MR device, selection of an imageof a second MR device visible in the display of the first MR device;sending, to the second MR device, a notification associated with sharingthe virtual content object, wherein the virtual content object is onlydisplayed on the first MR device; receiving, from the second MR device,a response to the notification associated with sharing the virtualcontent object; and sending, to the second MR device, the virtualcontent object based on the response to the notification.
 43. Anapparatus comprising: a processor; and a non-transitorycomputer-readable medium storing instructions operative, when executedby the processor, to cause the apparatus to: enable, at a first mixedreality (MR) device, a user of the first MR device to select a virtualcontent object displayed by a display of the first MR device; enable, atthe first MR device, the user of the first MR device to select an imageof a second MR device visible in the display of the first MR device,wherein the selected virtual content object was not previouslyaccessible to the second MR device; and responsive to receiving a userselection of the virtual content object and a user selection of theimage of the second MR device, make the virtual content objectaccessible to the second MR device.
 44. The apparatus according to claim43, further comprising: a touchscreen configured to: display a renderedmixed reality scene, with the virtual content object to be sharedrendered onto the mixed reality scene, and detect a user input, whereinthe user input is a gesture performed by a user on the touchscreen. 45.An apparatus comprising: a processor; and a non-transitorycomputer-readable medium storing instructions operative, when executedby the processor, to cause the apparatus to: enable, at a first mixedreality (MR) device, selection of a virtual content object displayed bya display of the first MR device; enable, at the first MR device,selection of an image of a second MR device visible in the display ofthe first MR device; send, to the second MR device, a notificationassociated with sharing the virtual content object, wherein the virtualcontent object is only displayed on the first MR device; receive, fromthe second MR device, a response to the notification associated withsharing the virtual content object; and send, to the second MR device,the virtual content object based on the response to the notification.46. An apparatus comprising: a processor; and a non-transitorycomputer-readable medium storing instructions operative, when executedby the processor, to cause the apparatus to: receive from a first mixedreality (MR) device information related to a virtual content in themixed reality scene which is selected by the first MR device, whereinthe received information comprises a 3D position of the selected virtualcontent; and render, onto the mixed reality scene, the selected virtualcontent with regard to the 3D position of the selected virtual contentand a viewing position and/or orientation of a second MR device.
 47. Amethod comprising: receiving from a first mixed reality (MR) deviceinformation related to a virtual content object in the mixed realityscene which is selected by the first MR device, wherein the receivedinformation comprises a 3D position of the selected virtual contentobject; and rendering, onto the mixed reality scene, the selectedvirtual content object with regard to the 3D position of the selectedvirtual content object and a viewing position and/or orientation of asecond MR device.